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Comments are special types of publications whose aim is to correct or criticize previously published 
papers. For this reason, comments are believed to make commented papers less worthy or trusty to 
the eyes of the scientific community, and thus predestined to have low scientific impact. Here, we 
show that such belief is not supported by empirical evidence. We consider thirteen major publica- 
tion outlets in science, and perform systematic comparisons between the citations accumulated by 
commented and non commented articles. We find that (i) commented papers are, on average, much 
more cited than non commented papers, and (ii) commented papers are more likely to be among the 
most cited papers of a journal. Since comments are published soon after criticized papers, comments 
should be viewed as early indicators of the future impact of criticized papers. 



I. INTRODUCTION 

In 1543, Nikolaus Copernicus proposed a heliocentric 
model that will have revolutionized the human view of 
the universe pQ . The paradigm shift proposed by the new 
ideas of Copernicus caused an unavoidable controversy in 
the scientific community of his era, still anchored to the 
geocentrism. The dispute involved some of the bright- 
est brains of the period -including Galilei, Kepler and 
Netwon- and lasted for more than two centuries before 
the complete acceptance of the heliocentric model for the 
the description of the solar system. The controversy be- 
hind the Copernican revolution is just one of the most 
popular examples of scientific controversies that are part 
of the history of science. Other well known examples 
are the controversy which followed the publication of the 
theory of evolution by Darwin [2], the Bohr- Einstein de- 
bate about the fundamentals of quantum mechanics [3], 
and the dispute originated by Wegener with his theory 
of continental drift [I]. In the course of the history of 
science, however, not all scientific disputes have been re- 
solved in favor of the original idea that caused the con- 
troversy: the observation of TV-rays [5] , the theory of cold 
fusion [6] and the finding of water memory [7J [5] are all 
examples of theories or experimental results, associated 
to fervent scientific controversies, that have been at the 
end rejected or disregarded by the scientific community. 

Either resolving in favor or against the scientific find- 
ings that originated the disputes, scientific controversies 
are thought to be necessary for scientific progress HO] • 
Even if not all the greatest achievements in science have 
passed through a dispute, as for example the unification 
of electricity and magnetism by Maxwell, many major 
steps in science have been controversial. Revolutionary 
changes are per se controversial because they reverse pre- 
vious scientific paradigms, and thus necessarily encounter 
some resistance before getting accepted. Scientific dis- 
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putes, however, are not only associated to revolutionary 
discoveries, but they are also part of the process of sci- 
entific production: science is, in fact, simultaneously a 
cooperative and antagonistic enterprise, where scientists 
both collaborate, with the interchange of information, 
and compete, through the exchange of criticisms. While 
the structure of collaboration networks |llj and the im- 
portance of teams for the creation of scientific knowl- 
edge [121 1 1 3] have been empirically analyzed, no much is 
quantitatively known about scientific disputes. Scientific 
controversies are usually studied in philosophy of science, 
but only through the analysis of popular case examples 
and never in quantitative terms [HI HO] . 
Here for the first time, we provide a quantitative and 
large-scale study of scientific controversies. We focus our 
attention on modern scientific disputes identifiable with 
the publication of formal comments. We systematically 
study the difference between the citations accumulated 
by commented and non commented papers, and show 
that comments can be statistically interpreted as early 
signs of the future impact of criticized papers. 



II. RESULTS 

Comments are short publications whose purpose is 
to address core arguments, theories or experimental 
results of recently published research papers. The 
name of this type of publications varies from journal 
to journal. For example, in Nature they are called 
"Brief Communications Arising" , in Science "Technical 
Comments" , in Physical Review Letters "Comments" , 
in New England Journal of Medicine "Letters to the 
Editor" , etc. When submitted, comments are considered 
by journal editors in a similar way as normal articles: 
in order to be published, they need to satisfy strict 
requirements of broad interest, and have to pass the 
scrutiny of peer-review. The vast majority of com- 
ments represent formal criticisms to the content of 
commented papers. Irrespectively of the journal of 
publication, editorial policies are, in this regard, very 
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explicit: just to cite an example, in Physical Review 
Letters "a comment corrects or criticizes a specific 
Letter" and "is not meant to be a vehicle for addenda" 
(forms.aps.org/autlior/comments-prl.pdf). In ad- 
dition, a comment is generally followed by a so-called 
"reply" or "response" , written by the same authors 
of the criticized paper, with the purpose of defending 
their own paper from the criticisms of the comment. 
Comments and replies are published one after the other 
in the same issue of the journal, and thus provide a fair 
way to give birth to a scientific dispute, where both 
parts present and defend their own opinions. 
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Figure 1: Commented papers accumulate on average more 
citations, a. Average number of citations received by papers 
published in Physical Review Letters as a function of the year 
of publication. The average number of citations accumulated 
by non commented papers is represented by the black line, 
while the average number of citations received by commented 
papers is represented by the gray bars, b, c and d. Same as in 
panel a but for articles published in Science, Water Resources 
Research and Geology, respectively. 

We identified all formal comments published in the last 
ten to fifty years in thirteen major publication outlets in 
science, including multidisciplinary and specialist jour- 
nals (focusing on research topics in environmental sci- 
ences, geology, medicine and physics). We automatically 
associated each comment to the criticized publication 
(see supplementary information). Finally, we collected 
from the Web Of Science database (isiknowledge.com) 
the number of citations accumulated by each publication 
(as of April 2012). In our citation analysis, we restricted 
the attention only to papers published before 2008 in 
order to rank papers on the basis of stable citation dis- 
tributions Q3] and be confident that the vast majority of 
comments to these papers have been already published 
(i.e., in more than the 95% of the cases, comments are 
published less than 4 years after criticized papers, see 
Figures 3 and S2-10). 

Though commented papers represent a small percentage 
of the publications of a journal (Table SI), already with 



a qualitative analysis it is possible to notice that many 
of the top cited articles of a journal are papers that were 
criticized in formal comments (Tables S2-4). In Phys- 
ical Review Letters for example, the most cited paper 
is "Generalized Gradient Approximation Made Simple" , 
Phys. Rev. Lett. 77, 3865 (1996), with over 20,000 cita- 
tions, and this paper was criticized in a formal comment. 
More in particular, while the percentage of commented 
papers in Physical Review Letters in our period of ob- 
servation is just 3%, we find that the 5 most cited com- 
mented papers are in the list of the 16 most cited papers 
of the journal, a proportion 9.4 times larger than what 
expected by chance. In the other journals, the situation 
is similar and the 5 most cited commented papers have 
an absolute rank, based on the raw number of citations 
they have accumulated, from 2 to 44 times higher than 
what expected in the case in which being or not being 
commented would be unrelated to the number of cita- 
tions accumulated (Table SI). The only less evident case 
is represented by the publications in Nature, where the 
rank of the 5 most cited commented papers of the journal 
is 1.1 times higher than what expected by chance. 

From the previous qualitative analysis based on a lim- 
ited number of case examples, it seems not only that 
commented papers receive more citations than non com- 
mented papers, but also that commented articles are un- 
expectedly more present in the population of the most 
cited publications. In order to statistically confirm the 
validity of these observations, we present here a system- 
atic analysis. 

First, we look at the average citation rates of commented 
and non commented papers. In order to avoid age- 
dependent biases in the number of citations |15j , we com- 
pare the average number of citations received by papers 
published in the same year. The results for the vari- 
ous journals are presented in Figures 1 and S2-10. We 
confirm indeed that commented papers are, on average, 
much more cited than non commented papers. How- 
ever, the observation that a commented paper typically 
receives more citations than a non commented article is 
valid not only by looking at average citation numbers, but 
also by considering other types of measures (e.g, median 
citation numbers, Figure Sll) and performing non para- 
metric statistical tests (e.g., Mann- Whitney-test, Fig- 
ure S12). In general, while in single years of publication 
the fact that a commented paper is more cited than a non 
commented article could be still explained in terms of sta- 
tistical fluctuations, the persistence over many years of a 
positive signal allows us to say that commented papers 
significantly accumulate more citations than non com- 
mented articles. 

Second, we investigate the presence of commented pa- 
pers in the population of the most cited articles. We 
compare only papers published in the same journal, and 
assign to each paper a score equal to the fraction of ar- 
ticles published in the same year that received a smaller 
number of citations. If the score of a paper is equal to 
one this means that the paper is the most cited publi- 
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Figure 2: Commented papers are more likely to be among the 
most cited publications of a journal. Fraction of commented 
papers (black circles) that are part of the top 5% of the most 
cited papers published in the journals considered in our anal- 
ysis. This fraction is calculated as C$%/C, with C$% number 
of commented papers within the top 5%, and C total number 
of commented papers. The quantity C^%/C thus represents 
the conditional probability that, given that a paper is com- 
mented, this paper is within the top 5%. We compare the 
measured fraction of commented papers within the top 5% 
with the distribution of these values calculated in a statisti- 
cal model where commented and non commented papers are 
equally likely to be in the top 5%. The horizontal lines inside 
the boxes denote the median value (50%) of the distribution, 
boxes delimit the 68.2% confidence intervals (i.e., one stan- 
dard deviation), and error bars denote the 95.4% confidence 
intervals (i.e., two standard deviations). 



or have not been commented. By generating 10, 000 in- 
dependent realizations, we find that commented papers 
over-populate the top 5% of cited papers in each journal 
(i.e., measured values are larger than expected median 
values), and more importantly that, in the majority of 
the journals, these proportions are at least two standard 
deviations larger than what one would expect by chance 
(Figure 2). 

More in detail, by looking at the rank probability den- 
sity of commented papers (Figure S13), we can observe a 
general pattern with the following properties: (i) At low 
rank positions, commented papers are less present than 
expected. This is not surprising since low rank positions 
are mainly occupied by uncited papers or papers with 
few citations, and commented papers are never uncited 
because they are cited at least by the comment and the 
eventual reply, (ii) At medium rank positions, the dis- 
tribution is compatible with the expected uniform distri- 
bution, (iii) At high rank positions, commented papers 
are more present than expected. This means that com- 
mented papers over-populate the set of the most cited 
publications of each journal, and this observation is sta- 
tistically incompatible with the assumption that com- 
mented and non commented papers accumulate citations 
with the same rates. Notice that this does not mean that 
a paper, in order to be highly cited, necessarily needs to 
be a commented paper, but that the conditional prob- 
ability to be a top cited paper, given that the paper is 
a commented article, is much higher than the respective 
unconditional probability. For example in Physical Re- 
view D, the unconditional probability to be in the top 5% 
is 0.05, while the conditional probability for commented 
papers is three times larger and equals 0.15. 



cation of the year, while if the paper's score equals zero 
this means that the paper is among the least cited ar- 
ticles of the year. Since the score is not dependent on 
the publication year (i.e., the score is a distribution- free 
indicator), we can directly compare the impact of pa- 
pers published in different years, and thus increase the 
overall number of comparisons and obtain a more clear 
statistical picture. We then perform, using these scores, 
a ranking between all papers independently of their year 
of publication, and select the top 5% of publications (or 
equivalently those with score larger than 0.95) in the 
ranking. Notice that this is equivalent to the selection 
of the top 5% of the most cited papers in each year. We 
finally count the fraction of commented papers in the 
top 5%, and compare the measured values with those 
expected by chance in the hypothesis that being in the 
top 5% and being a commented paper would be two in- 
dependent events |16j . A realization of this statistical 
ensemble is obtained by randomly mixing the scores only 
among papers published in the same year. This proce- 
dure ensures that the number of commented papers per 
year is constant, but removes any eventual dependence 
between citation numbers and the fact that papers have 



III. DISCUSSION 

Contrarily to the popularity of the wisdom of "any 
publicity is good publicity", according to which success 
might follow from negative criticisms, there are very few 
empirical validations of this belief [T7] . In this sense, it 
is surprising that one example, probably the most clear 
empirical observation so far, is indeed offered by science. 
It should be noticed that our analysis does not include 
all the possible ways of criticizing previously published 
articles. Our approach is in fact limited only to the 
case of explicit comments published in the same journal 
of publication which published the original, criticized, 
article, and therefore neglects possible cases of "implicit" 
criticisms arising from the publication of other regular 
articles. This clearly precludes the application of the 
same type analysis to disciplines where the publication 
of formal comments is not practiced. 
Also, since comments and commented articles are 
published at short time distance (Figures 3 and S2-10), 
it is difficult to make claims about the causality effect of 
a comment on the number of citations accumulated by 
a commented paper. The time gap is in fact too short 
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Figure 3: Critical comments follow papers after a short time, 
a. Probability density function of the difference in publica- 
tion dates between comments and commented papers in Phys- 
ical Review Letters. Comments are, on average, published 
t — 13.5 months later than commented papers (standard de- 
viation o T = 11.8). In the inset, we report the cumulative 
distribution function of the difference in publication dates be- 
tween comments and commented papers in Physical Review 
Letters. The 95% of the comments are published 4 — 5 months 
later than criticized papers, but also in less 30 — 31 months af- 
ter the publication of commented papers, b. The typical dif- 
ference between the publication dates of comments and com- 
mented papers in Science is r = 8.2 (o> = 3.7). In the 95% 
of the cases, comments in Science are published 3 — 4 months 
after the publication of the commented papers, and before 
14 — 15 months since the criticized papers have been pub- 
lished, c. In Water Resources Research, the time difference 
between the publication dates of comments and commented 
papers has an average value r = 18.3 (a T — 11.8). In the 95% 
of the cases, commented papers in Water Resources Research 
receive a comment after 9 — 10 months but also before 36 — 40 
months since their publication, d. In Geology, papers are 
commented after an average period r = 10.1 (a T = 5.4). In 
the 95% of the cases, comments and commented papers have 
publication dates that differ more than 4 — 5 months, but also 
less than 17 — 18 months. 



to monitor eventual differences in the trends of accu- 
mulation of citations before and after the publication of 
a comment, and this does not allow us to understand 
whether highly cited papers attract comments or instead 
comments generate citations. Our intuitive interpreta- 
tion is the following. We think that the potentiality of 
a paper to attract future citations increases the chances 
that the paper gets commented. Scientists are, in fact, 
able to early recognize papers that are predestined to 
be highly influential, and thus these papers are more 
likely to be deeply examined and eventually commented. 
We also think that important scientific discoveries 
often originate controversies, and the publication of a 
comment is a way of initiating a dispute. This leads to 
boosts of attention and consequently to higher rates of 
citation accumulation. Independently of the possible 
interpretation, our results reveal a coherent pattern for 



all the scientific disciplines considered here. On the one 
hand, comments can be viewed as negative labels for 
criticized papers because comments arc published to 
criticize or correct other papers. On the other hand, 
comments can be also viewed as positive labels for 
criticized papers because they statistically represent 
early indicators of the future impact of criticized articles. 
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Figure 4: The publication rate of comments has drastically 
decreased in recent years. Relative number of commented 
papers in Physical Review Letters (a), Science (b), Water 
Resources Research (c) and Geology (d), as a function of the 
year of publication. 

In all journals for which we could monitor the publica- 
tion of comments over sufficiently long time windows, we 
realized that, in the latest ten to fifteen years, the pro- 
portion of commented articles has significantly decreased 
(Figures 4 and S2-10). It seems therefore that, in the 
process of creation of scientific knowledge, scientists are 
increasingly preferring to avoid scientific "fights" . There 
could be multiple reasonable explanations for the obser- 
vation of such decrement in the rate of publication of 
comments, and, here, we list only few of them, (i) Sci- 
entific teams are increasingly dominating the process of 
production of knowledge, and thus, even if not mutually 
exclusive, collaboration is overwhelming antagonism, (ii) 
Writing a comment represents an investment of time and 
effort as much as a normal article, but has low benefit 
for the academic curriculum of the authors of the com- 
ment and can be potentially dangerous for the creation 
of scientific "enemies" . (iii) The number of scientific pa- 
pers published today is too large. Thus, either scientists 
have not enough time to read carefully all published doc- 
uments and eventually commenting on them, or the num- 
ber of central topics in science that deserve the birth of 
a controversy is not growing at the same rate, (iv) The 
process of peer-review has become so precise and efficient 
that the percentage of published papers potentially crit- 
icizable has decreased. 

Clearly, a simple citation analysis, as the one performed 
here, cannot provide an exact clue of the reasons for this 
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evident reduction. Our study just represents a starting 
empirical observation from which additional questions 
might arise, and further investigations are required in 
order to understand more deeply such phenomenon. Our 
analysis, however, provides already a quantitative recom- 
mendation to both scientists and journal editors to con- 
sider the fundamental importance of an open scientific 
discussion for the progress of scientific knowledge. If a 
paper receives a comment, this does not necessarily rep- 
resent a negative event for the journal which published 
the paper or for the scientists who wrote the paper, but 



could be instead an early indication of the importance of 
the paper itself. As we demonstrated in fact, if citation 
numbers truly reflect the scientific impact of a publica- 
tion (although this statement is also under debate [T5J- 
I21|). then for a paper it is better being commented than 
not being commented. Indeed, what Oscar Wilde wrote 
in the Picture of Dorian Gray about gossip seems to be 
valid also in science: "there is only one thing in the world 
worse than being talked about, and that is not being 
talked about." 
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SUPPLEMENTARY INFORMATION 
Data collection 

Physical Review 

For the publication period 1958 — 2009, we 
used the data set directly provided by the edito- 
rial office of the American Physical Society (APS, 
publish.aps.org/datasets). We identified potential 
comments as those publications which satisfy one of the 
following criteria: 

1. publications whose title starts with the word "Com- 
ment" ; 

2. publications classified as "comments" in the APS 
database, but with title not containing the words 
"Reply" or "Response". 

We then verified for each element in the list of potential 
comments whether or not they were effectively comments 
to previously published papers in journals of the Physical 
Review's collection. We used two ways to determine this 
fact and also to associate real comments to commented 
papers: 

1. we parsed the web page of the potential comment, 
and searched for the associated commented article 
(the criticized article is referenced in this page as 
"Original Article"); 

2. when the previous information was not present, we 
read the content of the potential comment and de- 
termined if the document was a real comment with 
an associated criticized paper. 

For years 2010 and 2011, we identified potential com- 
ments as all documents with title starting with "Com- 
ment on" (the search was performed in the search engine 
of the APS website). We then identified real comments 
and associated commented articles in the same way as 
the one described above. 

In general, we found that comments were criticizing pa- 
pers published in the same journal. We found also a 
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small percentage of comments criticizing papers pub- 
lished in other Physical Review journals. Mainly com- 
ments to Physical Review Letters articles published in 
the period 1970—1979 in Physical Review A-D, and com- 
ments to Physical Review A articles published in the pe- 
riod 1993 - 1994 in Physical Review E (in 1993 Physical 
Review A was split in Physical Review A and in the newly 
created journal Physical Review E). 

Science 

We identified all comments as the elements published 
by Science in the journal section "Technical Comments" 
(this list was retrieved at 

www . sciencemag . org/ cgi/ collection/tech_comment 

for publications since 1999, and by reading the content 
of each issue for the previous years). We were able 
to consider only publications since 1997 because the 
electronic journal archive covers publications from 1997 
on (for previous years, publications are provided only 
in pdf format). We associated each comment to the 
criticized paper by parsing the web page of the on-line 
version of the comment and finding the associated doc- 
ument (the criticized paper is listed after the sentence 
"The editors suggest the following Related Resources on 
Science sites"). When this information was not present, 
we instead read the text of the comment. 



Nature 

For publication year 2004 and after, we identified 
all comments as the elements published in Nature as 
" Brief Communications Arising" (such list was obtained 
by parsing the content of all issues of the journal at 
www.nature.com). For years 1999 to 2003, comments 
have been instead identified (by reading the text of the 
publications) among those papers published in the sec- 
tion " Brief Communications" . We were able to consider 
only publications since 1999 because the electronic jour- 
nal archive covers publications from 1999 on (for previ- 
ous years, publications are provided only in pdf format). 
We associated each comment to the criticized paper by 
parsing the web page of the on-line version of the com- 
ment and finding the associated document (the reference 
to criticized paper appears at the beginning of the com- 
ment after the sentence "Arising from"). When this in- 
formation was not present, we instead read the text of 
the comment. 



New England Journal of Medicine 

We identified as potential comments all the elements 
published by New England Journal of Medicine in the 
section "Correspondence" (the entire publication list of 
the correspondence section can be retrieved with the 



search engine offered at www.nejm.org). We were able 
to consider only publications since 1990 because the elec- 
tronic journal archive covers publications from 1990 on 
(for previous years, publications are provided only in pdf 
format). We automatically parsed the content of all po- 
tential comments, and identified true comments as those 
publications in which we identified a string of the for- 
mat " (month day issue) reference" . The information con- 
tained in this string was also used to identify the criti- 
cized papers. 

Journal of Chemical Physics 

We identified all comments as those publication with 
title of the format "Comment on {title of the commented 
paper} [J. Chem. Phys. vol., page (year)]". We limited 
our attention only to publications after 1999 because in 
previous years we were not able to detect a regular pub- 
lication rate of comments. The reference to the criticized 
papers was obtained by parsing the titles of the com- 
ments. Data have been collected from the journal web 
site (jcp.aip.org). 

Geology 

We identified all comments as those publications with 
titles in the format "Comment on {title of the com- 
mented article}" or "title of the commented article - 
Comment". Criticized articles where automatically de- 
tected by matching their titles with those appearing in 
the titles of the comments. Data have been collected 
from the journal web site (geology.gsapubs.org). 

Environmental Science & Technology and Water Resources 
Research 

We identified all comments as those publications with 
titles in the format "Comment on {title of the com- 
mented article}" or "title of the commented article - 
Comment" . Criticized articles where automatically de- 
tected by matching their titles with those appearing in 
the titles of the comments. Data in this case have been 
retrieved from the Web of Science database. 
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Journal 


Period of observation 


N 


C 


Pc 


T 


r c 


Environ. Sci. & Technol. 


1981 - 


2007 


13,626 


206 


1.51% 


152 


2.3 


Geology 


1975 - 


2007 


7,498 


723 


9.64% 


9 


5.8 


J. Chem. Phys. 


1999 - 


2007 


23,371 


111 


0.47% 285 


3.7 


Nature 


1999 - 


2007 


8,956 


182 


2.03% 


233 


1.1 


New Engl. J. Med. 


1990 - 


2007 


5, 082 2, 667 52.48% 


5 


1.9 


Phys. Rev. A 


1970 - 


2007 


47, 195 


491 


1.04% 


96 


5.0 


Phys. Rev. B 


1970 - 


2007 123,242 


748 


0.61% 


132 


6.2 


Phys. Rev. C 


1970 - 


2007 


27,203 


248 


0.91% 


176 


3.1 


Phys. Rev. D 


1970 - 


2007 


48,816 


370 


0.76% 


15 44.0 


Phys. Rev. E 


1993 - 


2007 


30,413 


209 


0.69% 195 


3.7 


Phys. Rev. Lett. 


1970 - 


2007 


74, 157 I 


!,475 


3.34% 


16 


9.4 


Science 


1997- 


2007 


9,762 


281 


2.88% 


74 


2.3 


Water Resour. Res. 


1966 - 


2007 


9,487 


368 


3.88% 


13 


9.9 



Table SI: Summary table. For each journal, we report from left to right: the name of the journal, our observation window, 
the number N of papers published, the number C of papers that received at least a comment, the percentage P c of commented 
papers, the rank T of the fifth most cited commented paper, and the number of commented papers in the top T most cited papers 
of the journal divided by the expected number of such commented papers in the hypothetical situation in which commented 
and commented papers would not differ in the ability to accumulate citations. The latter number is simply calculated as 
r c = (5 N) I (T C) since (t) = (TC) /N is the expected number of commented articles in the top T most cited papers of the 
journal. 



rank citations reference 


commented in 


1 


21, 224 Phys. Rev. Lett. 77, 3865 (1996) 


80, 890 (1998) 


3 


6,968 Phys. Rev. Lett. 56, 930 (1986) 


57, 3235 (1986) 


5 


5, 540 Phys. Rev. Lett. 55, 2471 (1985) 


56, 2656 (1986) 


12 


3,946 Phys. Rev. Lett. 85, 3966 (2000) 


87, 249701 (2001) & 87, 249703 (2001) 


16 


3,331 Phys. Rev. Lett. 59, 381 (1987) 


62, 110 (1989) 


2 


6, 188 Science 286, 509 (1999) 


287, 2115 (2000) 


11 


3,644 Science 280, 69 (1998) 


281, 883 (1998) 


53 


2, 085 Science 290, 2319 (2000) 


295, 7 (2002) 


60 


1,956 Science 282, 1318 (1998) 


287, 1363 (2000) 


74 


1,690 Science 299, 1719 (2003) 


307, 1203 (2005) 


2 


1, 730 Water Resour. Res. 16, 574 (1980) 


17, 768 (1981) 


4 


1,032 Water Resour. Res. 14, 601 (1978) 


15, 989 (1979) 


5 


986 Water Resour. Res. 19, 161 (1983) 


19, 1641 (1983) 


8 


645 Water Resour. Res. 22, 2069 (1986) 


24, 315 (1988) & 24, 892 (1988) 
& 24, 1209 (1988) 


13 


525 Water Resour. Res. 11, 725 (1975) 


13, 477 (1977) 


4 


521 Geology 19, 598 (1991) 


20, 191 (1992) 


5 


520 Geology 19, 547 (1991) 


20, 475 (1992) 


6 


517 Geology 10, 70 (1982) 


11, 428 (1993) 


7 


493 Geology 19, 425 (1991) 


20, 87 (1993) 


9 


484 Geology 14, 99 (1986) 


14, 1042 (1986) 



Table S2: Top cited papers that received a comment. We report here the five top cited papers, among those that received at 
least one comment, published in Physical Review Letters in the period 1970 — 2007, in Science between 1997 and 2007, in Water 
Resources Research in the period 1966 — 2007 and in Geology between 1975 and 2007. For each paper, we report the following 
information: absolute rank based on the raw number of citations accumulated (the ranking includes all publications, either 
commented and not commented), total number of citations accumulated, reference of the commented publication, reference of 
the comment(s). 
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rank citations reference 


commented in 


32 


743 Phys. Rev. A 29, 2765 (1984) 


32, 3135 (1985) 


49 


609 Phys. Rev. A 39, 1665 (1989) 


43, 2576 (1991) 


65 


533 Phys. Rev. A 16, 531 (1977) 


26, 3008 (1982) 


84 


479 Phys. Rev. A 41, 2295 (1990) 


43, 5165 (1991) 


96 


447 Phys. Rev. A 26, 2028 (1982) 


36, 5463 (1987) 


3 


11,090 Phys. Rev. B 13, 5188 (1976) 


18, 5897 (1978) 


16 


2, 557 Phys. Rev. B 26, 4199 (1982) 


37, 4795 (1988) 


31 


1,693 Phys. Rev. B 12, 2455 (1975) 


16, 4719 (1977) 


95 


917 Phys. Rev. B 4, 3184 (1971) 


6, 311 (1972) & 6, 3546 (1972) 


132 


813 Phys. Rev. B 20, 624 (1979) 


22, 1095 (1980) 


24 


420 Phys. Rev. C 13, 1226 (1976) 


16, 885 (1977) 


65 


277 Phys. Rev. C35, 1678 (1987) 


37, 892 (1988) 


117 


207 Phys. Rev. CI, 769 (1970) 


5, 1135 (1972) 


141 


194 Phys. Rev. C 15, 1359 (1977) 


18, 573 (1978) 


176 


177 Phys. Rev. C9, 1018 (1974) 


12, 686 (1975) 


2 


2,916 Phys. Rev. D 10, 2445 (1974) 


12, 3343 (1975) 


3 


2,610 Phys. Rev. D 2, 1285 (1970) 


3, 1043 (1971) & 4, 1918 (1971) 


4 


2, 573 Phys. Rev. D 7, 1888 (1973) 


9, 1129 (1974) & 11, 2332 (1975) 






& 11, 3040 (1975) 


7 


1,984 Phys. Rev. D 9, 3471 (1974) 


12, 923 (1975) & 12, 4006 (1975) 


15 


1, 521 Phys. Rev. D 14, 870 (1976) 


18, 609 (1978) 


32 


354 Phys. Rev. E 68, 011306 (2003) 70, 043301 (2004) 


72 


274 Phys. Rev. E 48, R29 (1993) 


51, 2669 (1995) 


131 


206 Phys. Rev. E 54, 3853 (1996) 


60, 1099 (1999) 


150 


196 Phys. Rev. E 65, 041903 (1974) 67, 063901 (2003) & 72, 063901 (2005) 


195 


165 Phys. Rev. E 50, 2064 (1994) 


53, 2992 (1996) 



Table S3: Top cited papers that received a comment. We report here the five top cited papers, among those that received at 
least one comment, published in Physical Review A-D in the period 1970 — 2007, and in Physical Review E between 1993 and 
2007. For each paper, we report the following information: absolute rank based on the raw number of citations accumulated 
(the ranking includes all publications, either commented and not commented), total number of citations accumulated, reference 
of the commented publication, reference of the comment (s). 
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rank citations reference 


commented in 


1 


2, 205 Environ. Set. Technol. 36, 1202 (2002) 


36, 4003 (2002) & 36, 4005 (2002) 






& 37, 1052 (2003) 


2 


950 Environ. Set. Technol. 32, 1549 (1998) 33, 369 (1998) 


74 


328 Environ. Set. Technol. 30, 2432 (1996) 31, 1577 (1997) 


122 


270 Environ. Set. Technol. 27, 961 (1993) 


28, 366 (1994) & 28, 367 (1994) 


152 


251 Environ. Set. Technol. 30, 881 (1996) 


30, 3132 (1996) 


1 


8,834 New Engl. J. Med. 329, 977 (1993) 


329, 1661 (1993) & 






3 comm. in 330, 641 (1994) 


2 


4,866 New Engl. J. Med. 333, 1301 (1995) 


3 comm. in 334, 1333 (1996) 


3 


4,827 New Engl. J. Med. 338, 853 (1998) 


339, 405 (1998) 


4 


4, 676 New Engl. J. Med. 346, 393 (2002) 


4 comm. in 346, 1829 (2002) 


5 


4, 339 New Engl. J. Med. 342, 145 (2000) 


4 comm. in 343, 64 (2000) 


24 


457 J. Chem. Phys. 114, 5149 (2001) 


116, 11039 (2002) 


198 


149 J. Chem. Phys. 123, 164110 (2005) 


124, 107101 (2006) 


218 


144 J. Chem. Phys. 119, 12784 (2003) 


121, 3347 (2004) 


250 


135 J. Chem. Phys. 119, 2376 (2003) 


120, 9427 (2004) 


285 


129 J. Chem. Phys. 122, 204302 (2005) 


125, 047101 (2006) 


90 


1,260 Nature 401, 82 (1999) 


406, 367 (2000) 


92 


1,231 Nature 434, 214 (2005) 


437, E3 (2005) & 437, E3 (2005) 


96 


1,211 Nature 427, 145 (2004) 


3 comm. in 430 issue of I s * July, 2004 


186 


940 Nature 425, 944 (2003) 


in 429 issue of 13 ih May, 2004 


233 


845 Nature 423, 280 (2003) 


434, El (2005) 



Table S4: Top cited papers that received a comment. We report here the five top cited papers, among those that received at 
least one comment, published in Environmental Science & Technology in the period 1981 — 2007, in New England Journal of 
Medicine between 1990 and 2007, in Journal of Chemical Physics in the period 1999 — 2007 and in Nature between 1999 and 
2007. For each paper, we report the following information: absolute rank based on the raw number of citations accumulated 
(the ranking includes all publications, either commented and not commented), total number of citations accumulated, reference 
of the commented publication, reference of the comment (s). 
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Figure SI: Number of commented papers and comments published in each year. a. Number of papers published in Physical 
Review Letters whose comment was published in the same journal (black circles) or in other Physical Review journals (blue 
diamonds) as function of the year of publication. Number of comments published in Physical Review Letters (red squares) as 
a function of the year of publication, b, c and d. Number of commented papers (black circles) and comments (red squares) 
published in Science, Water Resources Research and Geology, respectively. 
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Figure S2: Analysis of the publications in Physical Review A. a. Number of comments (red squares) and commented papers 
(black circles) as functions of the year of publication, b. Fraction of commented papers as a function of the year of publication, 
c. Average number of citations accumulated by papers published in a given year. Average citation numbers of commented 
papers (gray bars) are compared to those of non commented papers (black line), d. Probability density function (main plot) 
and cumulative distribution (inset) of the time difference between the publication dates of comments and commented papers. 
On average, comments are published r = 22.1 months after commented articles (standard deviation ay = 21.4). 
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Figure S3: Analysis of the publications in Physical Review B. a. Number of comments (red squares) and commented papers 
(black circles) as functions of the year of publication, b. Fraction of commented papers as a function of the year of publication, 
c. Average number of citations accumulated by papers published in a given year. Average citation numbers of commented 
papers (gray bars) are compared to those of non commented papers (black line), d. Probability density function (main plot) 
and cumulative distribution (inset) of the time difference between the publication dates of comments and commented papers. 
On average, comments are published r = 21.6 months after commented articles (standard deviation o> = 19.4). 
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Figure S4: Analysis of the publications in Physical Review C. a. Number of comments (red squares) and commented papers 
(black circles) as functions of the year of publication, b. Fraction of commented papers as a function of the year of publication, 
c. Average number of citations accumulated by papers published in a given year. Average citation numbers of commented 
papers (gray bars) are compared to those of non commented papers (black line), d. Probability density function (main plot) 
and cumulative distribution (inset) of the time difference between the publication dates of comments and commented papers. 
On average, comments are published r = 16.5 months after commented articles (standard deviation a T — 20.8). 
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Figure S5: Analysis of the publications in Physical Review D. a. Number of comments (red squares) and commented papers 
(black circles) as functions of the year of publication, b. Fraction of commented papers as a function of the year of publication, 
c. Average number of citations accumulated by papers published in a given year. Average citation numbers of commented 
papers (gray bars) are compared to those of non commented papers (black line), d. Probability density function (main plot) 
and cumulative distribution (inset) of the time difference between the publication dates of comments and commented papers. 
On average, comments are published r = 20.1 months after commented articles (standard deviation ay = 19.6). 
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Figure S6: Analysis of the publications in Physical Review E. a. Number of comments (red squares) and commented papers 
(black circles) as functions of the year of publication, b. Fraction of commented papers as a function of the year of publication, 
c. Average number of citations accumulated by papers published in a given year. Average citation numbers of commented 
papers (gray bars) are compared to those of non commented papers (black line), d. Probability density function (main plot) 
and cumulative distribution (inset) of the time difference between the publication dates of comments and commented papers. 
On average, comments are published r = 21.8 months after commented articles (standard deviation a T — 16.4). 
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Figure S7: Analysis of the publications in Environmental Science & Technology, a. Number of comments (red squares) and 
commented papers (black circles) as functions of the year of publication, b. Fraction of commented papers as a function of 
the year of publication, c. Average number of citations accumulated by papers published in a given year. Average citation 
numbers of commented papers (gray bars) are compared to those of non commented papers (black line), d. Probability density 
function (main plot) and cumulative distribution (inset) of the time difference between the publication dates of comments 
and commented papers. On average, comments are published r = 9.6 months after commented articles (standard deviation 
a T = 6.5). 
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Figure S8: Analysis of the publications in New England Journal of Medicine, a. Number of comments (red squares) and 
commented papers (black circles) as functions of the year of publication, b. Fraction of commented papers as a function of 
the year of publication, c. Average number of citations accumulated by papers published in a given year. Average citation 
numbers of commented papers (gray bars) are compared to those of non commented papers (black line), d. Probability density 
function (main plot) and cumulative distribution (inset) of the time difference between the publication dates of comments 
and commented papers. On average, comments are published r = 4.8 months after commented articles (standard deviation 
a T = 3.0). 
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Figure S9: Analysis of the publications in Journal of Chemical Physics, a. Number of comments (red squares) and commented 
papers (black circles) as functions of the year of publication, b. Fraction of commented papers as a function of the year of 
publication, c. Average number of citations accumulated by papers published in a given year. Average citation numbers of 
commented papers (gray bars) are compared to those of non commented papers (black line), d. Probability density function 
(main plot) and cumulative distribution (inset) of the time difference between the publication dates of comments and commented 
papers. On average, comments are published r = 17.6 months after commented articles (standard deviation a T — 26.3). 
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Figure S10: Analysis of the publications in Nature, a. Number of comments (red squares) and commented papers (black 
circles) as functions of the year of publication, b. Fraction of commented papers as a function of the year of publication, c. 
Average number of citations accumulated by papers published in a given year. Average citation numbers of commented papers 
(gray bars) are compared to those of non commented papers (black line), d. Probability density function (main plot) and 
cumulative distribution (inset) of the time difference between the publication dates of comments and commented papers. On 
average, comments are published r = 13.1 months after commented articles (standard deviation a T = 11.6). 
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Figure Sll: Median number of citations as a function of the publication year of papers. The black line is calculated for 
commented papers, while the red line for non commented ones. The median value of citations accumulated by commented 
papers is compared with what expected by chance in the case in which comments are attributed randomly to papers. Boxes 
delimit the 68.2% confidence intervals (i.e., one standard deviation), while error bars denote the 95.4% confidence intervals 
(i.e., two standard deviations). 
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Figure S12: Probability that a commented paper accumulated at least the same number of citations of a non commented paper 
(u-test). Comparisons are made only between papers published in the same year. Values of the u-test are compared with those 
expected by chance in the case in which comments are attributed randomly to papers. Boxes delimit the 68.2% confidence 
intervals (i.e., one standard deviation), while error bars denote the 95.4% confidence intervals (i.e., two standard deviations). 
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Figure S13: Probability density of commented papers belonging to a given percentage bin of top cited papers (red curve). We 
used bin length equal to 5%. As a term of comparison, we show also the expected confidence intervals in the case in which 
comments are attributed randomly to papers. The black line show the median value expected in this case, the light gray region 
denotes the 68.2% confidence interval (i.e., one standard deviation), while the dark gray region denotes the 95.4% confidence 
interval (i.e., two standard deviations). 



